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ABSTRACT 

A Web-Based Learning Portfolio (WBLP) was evaluated through 
practical teaching process to understand if the WBLP system helps students to 
grasp the learning process and enhances learning outcomes . The evaluation 
results reveal that this WBLP system has been more useful for students to 
obtain the feedback from other students than from their teachers. It reflects 
the fact that the feedback from other students has been helpful for them and 
has thus become the necessary component to help them in learning. Whether the 
use of the Web-based portfolio system will definitely enhance students* self- 
regulated learning, self-directed learning, ability for self-reflection, and 
learning motivation are issues for further research and verification. 
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Abstract: A Web-Based Learning Portfolio (WBLP) was evaluated through practical teaching proc- 
TO THE EDUCATIONAL RESOURCES ^ ^ r u n u i j i j u 

INFORMATION CENTER (ERIC) understand if the WBLP system helps students to grasp the learning process and aihances 

1 learning outcomes. The evaluation results reveal that this WBLP system has been more useful for 

them to obtain the feedback from other students than from their teachers. It reflects the fact that 
the feedback from other students has been helpful for them and has thus become the necessary com- 
ponent to help them in learning. Next step, we will conduct the further empirical study in terms of 
learning effects for the system. 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



Research Background 

Portfolio has been used to gather the learning activities of students in various aspects as well as the works of students with one 
purpose in mind. It allows the teachers, students, or their parents to understand and evaluate the learning process, improvement situa- 
tion, and academic achievement of students. It may also be used as evidence for the pupils to reflect on their learning and the changes 
in their comprehension during the study process. Portfolio may provide the following benefits (Hewitt, 1995): 

1 . It may demonstrate students' growth and improvement situation. 

2. It may encourage students to set up learning goals. 

3. It may provide the hard evidence concerning students' efforts. 

4. It may show students' performance or works. 

5. It may serve the purpose for job application or school application. 

6. It may help the faculty to review students' learning progress. 

7. It may serve to inspire the teacher and understand students’ performance. 

8. It stimulates students' introspective thinking and enhances self-assessment (Smith & Tillema, 1998; Wade & Yarbrough, 1996; Car- 
roll, Potthoff & Huber, 1996; Vavrus, 1990). 

9. It encourages students' learning interest, builds up students' self-confidence, and assists students to know more about themselves, and 
develops writing skill as well. 

10. It encourages students' participation in cooperation, and increases their self-esteem (Mullin, 1998). 

Portfolio method emphasizes that a learner participates actively and thus establishes the learning results of his/her own concern. 
Therefore, its development process whether it be instructed by teachers or established by students’ own accord, shall both attain the 
goal of encouraging the learners to be responsible for his/her own learning and to grasp all the goals of learning activities. Furthermore, 
portfolio assessment was originally designed as an authentic assessment approach meant for the improvement of the traditional pencil- 
paper tests. Thus, the assessment is real and active. In contrast with the traditional way of assessment, portfolio assessment pays at- 
tention not only to the results but also the processes involved. Since learning, teaching, and evaluation comprise of the integrated 
activities that shall go hand in hand, hence portfolio approach is actually a better way to grasp the complexity of learning processes. It 
also allows the learners or teachers’ introspection to take place. It may also help to reflect on the overall learning and teaching proc- 
ess, so as to give feedback and offer suggestions, thus to benefit the learning in the genuine way. In general, portfolio possesses the 
following characteristics or features (Wade & Yarbrough, 1996); 

1. Developmental: portfolio represents a certain period of students’ growth and learning. Portfolio is the long-term accumulated learn- 
ing results, not meant for short-term goals. Hence, it is actually an on-going process in its development. 

2. Dual-valued: portfolio offers both the teachers and students the value of two-way interaction. It allows a learner the opportunity to 
reflect and record his/her own learning process. It also offers teachers a good approach or method to evaluate students’ growth and 
achievements. 

3. Selective; portfolio offers students the opportunity to choose, so that students may self-determine what kind of contents that they 
want to put in the portfolio and how to organize the whole portfolio. It also allows students to set up the standard or basis for 
evaluation/ assessment. 

4. Authentic; portfolio incorporates the genuine works or performances of students. Traditional tests normally cannot reflect a student 
growth or potential in all aspects, but portfolio may demonstrate solid learning results and thus offers an authentic way of learning. 

5. Reflective: portfolio may reveal the evidence of self-reflection. Thus learners may review his/her own work, then to set up further 
goals through such a reflection. It may also help to review the previous efforts, and compare them with the actions afterward, for 
better understanding of his/her improvements or growth. 

6. Individual: portfolio is a personal learning and growing record based on individual ways of choices and organization. Consequently, it 
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reflects personal contents and style in students’ personal portfolio. 

7. Interactive: a learner may share with his/her teachers and peers through personal portfolio, and thus seeking guidance and sugges- 
tions. In this way the development and establishment of portfolio may be regarded as the process of cooperative interactions. 

Other than that, portfolio also possesses the characteristics of self-regulated learning and self-directed learning (Fischer & King, 
1995; Smith & Tillema, 1998), this is due to the fact that a portfolio allows the learner to self-determine his/her learning goals, 
strategies, and contents and then records them. Thus the role of scholars is merely to assist and provide feedbacks. That is why it would 
encourage a learner to become an independent, self-directed person through the development of personal portfolio. 

The Web-Based Learning Portfolio (WBLP) that has been structured through web interface and also has gone through the dif- 
ferent stages of analysis, design, development, and test is now being established. The effects of the WBLP in students’ learning are the 
issue worthy of our analysis and investigation. With the above pre-requisite in mind, this research hopes to achieve the following goals: 

1 . To assess the perceived effects of the WBLP, and analyze its impact on students’ learning process. 

2. To propose the suggestions for future research direction based on the results of evaluation. 



Web-Based Learning Portfolio 

This WBLP is mainly used to provide students with a web-based user environment. It would help students to complete their in- 
dividual portfolios through simple, easy-to-use interface for necessary guidance. It also allows the teachers and students to browse their 
classmates’ portfolios, and to give their feedback or assessment opinions. The contents of this WBLP include the following seven 
major items: the basic information of students, the learning goals of students, the works of students in a course, the records of students’ 
self-reflection or self-assessment, records of teachers’ feedback and assessment, records of peer feedback and assessment, personal web 
page of students. In addition to the afore-mentioned, in order to support students to create and browse the portfolios, as well as to 
facilitate and meet the needs for teachers to inspect and evaluate students’ portfolios, the functional areas of WBLP system consists of 
the following eight major items: Portfolio Browse, Portfolio Creation, Portfolio Guide, Portfolio Discussion Board, Portfolio Bulletin 
Board, Portfolio Suggestion Board, Student Data Maintenance, System Management (Chang, 2000; Chang, 2001). These functional 
areas mainly provide the following functions to the portfolio (as shown on Figure 1): 

1 . The function for students to conduct the various productions of portfolio contents. 

2. The function for teachers and students to browse the portfolios of other students or peers. 

3. The feedback and assessments of teachers as well as the feedback and mutual assessment mechanism among students themselves. 

4. The on-line guidance and illustration for students to create portfolio contents. 

5. The searching and management function of students’ personal information. 

6. The asynchronous discussion channel of portfolio creation process. 

7. The opinion feedback channel for students in using the system. 

8. The updated system announcement or course information concerned. 

9. The function for teachers to manage student information and announcement information. 



Design of the Research 

We use an established Web-Based Learning Portfolio (WBLP) on an undergraduate course in university. Through practical teach- 
ing process, we may evaluate if the WBLP helps students to grasp the learning process, if it enhances learning outcomes. The evalua- 
tion consists of user evaluation, user in-depth interviews, and expert evaluation. The evaluation also helps to understand the difficulties 
and problems of this system in its application and its possible impact on students’ learning. 

The user evaluation method had been conducted one and half month after the WBLP system implementation. It was carried out 
through the evaluation questionnaire designed by the researcher. The questionnaire was distributed in several copies to a class of 35 
students who were taking the course of “Computer and Instruction” in a Pre-service Teacher Education Program. This research has 
designed its own questionnaire for evaluation (as shown on Table 3). The questionnaire primarily asked the questions of the WBLP 
impact on the learning process. The contents of this questionnaire are based on relevant literatures and discussions of scholars and 
experts. They have gone through the process of examination for validity and reliability. The questions have been presented based on 
Likert’s five-point scaling method. That means the choice for answer is: strongly agree, agree, average, disagree, and strongly disagree. 

In order to understand the WBLP impacts on students’ learning in depth, so we supplement the study through the researcher 
self-designed “Questions Listing of User In-Depth Interviews” after the completion of user evaluation questionnaire analysis. Inter- 
views in-depth would then follow up to get to know the opinions and suggestions of the users concerning the uiresolved answers and 
controversial issues that could never be revealed through the previous questionnaire. Targeting for the unresolved questions and the 
controversial issues of the previous questionnaire of user evaluation, in-depth interviews were then conducted on 5 students selected 
randomly after the completion of questionnaire analysis work in order to deeply understand the users’ opinions and suggestions. 

Three experts were being invited to conduct the expert evaluation (including the course instructor, a portfolio assessment 
expert, and a web technology expert) to use this WBLP on-line. The evaluation work was then conducted through interview method 
after one week. In terms of expert evaluation, interviews were conducted based on the researcher’s self-designed “ Questions Listing of 
Expert Interviews”. In this way that we got to understand the expert opinions in regarding the WBLP impact on learning and teaching. 
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Validity and Reliability of Questionnaire 

Validity refers to whether a tool for measurement/survey that may achieve its intended functions or purpose. The questionnaire 
of user evaluation in this research has gone through the processes of pre-test and pilot-study to verify the validity of the questionnaire. 

For attaining the goal of questionnaire validity, it was necessary to consult two expert/scholars of portfolio assessment and web 
technology, and one teacher who instructed the course after the completion of the questionnaires for their opinions on revising the 
framework of said contents, words and expressions relating to the issues, etc. The suggestions for revising the questionnaire include; 
increasing the contents of the questionnaire, more articulate in the expressions of questions, word polishing, etc. Then could the con- 
tents of the questionnaire be able to attain the goal originally intended for the effectiveness in its measurement/survey. Through the 
implementation of such a method, this research meets the needs of measurement/survey tool efficiency. The above process involved is 
the so-called expert validity. 

We may achieve more complete validity verification through the further process of pilot-study. We picked up 5 students ran- 
domly for the purpose of this research at that time to conduct pilot-study based on the rough draft of the questionnaire. We hoped to 
go through such a pilot-test and render the results to the researcher to get to know if the questionnaire could effectively meas- 
ure/survey the purpose intended for this research, and to revise any inappropriateness whatever necessary. The opinions or feedback 
for revision include the clarification of some questions and word polishing, etc., so as to present the questions in the way better inder- 
stood by students who were answering the questions in the questionnaire. 

Reliability refers to being consistent and steady in terms of measurement/survey method. In this way we could understand the 
extent or degree of reliability for measurement/survey instruments. In order to attain the reliability goal of this questionnaire, we con- 
ducted the reliability test of Cronbach’s alpha coefficient by using the contents of the above retrieval questionnaires. The extent or 
degree of consistency within the questionnaires could thus be determined. Then we could determine if the research measurement tool 
possessed certain degree of reliability. The Cronbach’s alpha coefficient of formal questionnaire after revision is equal to 0.8346 (n = 
35). This shows that the reliability level of this research has attained a rather reliable level. 



Results and Discussions 
User Evaluation 

User evaluation consists of survey and interviews. Tables 1 list the statistics results of each evaluation item within the question- 
naire. Among the 25 evaluation items of Table 1, other than the later debut of the Discussion Board (came out only about one week 
prior to the evaluation) that has led to the insufficient understanding and use of its function, and thus resulted in the mean of this two 
items both lower than 4 (3.83 & 3.87 respectively) for Item 16 — the articles of Portfolio Discussion Board enhance my growth in 
learning and Item 17 — the feedback on the Portfolio Suggestion Board have been helpful for me to overcome the difficulty in produc- 
ing personal portfolio. As for the other 23 items of survey, the degree of agreeability mean is higher than 4, and most users are either 
agree with the statement or very much agree with it. The means of 6 items (Item, 12, 18, 19, 20, 24, 25) are even higher than 4.5, 
and 90% of the users regard them very helpful for their learning. 

From the survey results, it appears to be that most of the users are rather agreeable that this system has been helpful for their 
learning process or learning outcomes, and thus have come out with positive appraisal for this system. As for the questions in regarding 
to whether the browsing of other students’ portfolio, learning goals, works, basic information, personal web page, etc. may help them 
in some way (from Item 6 to Item 14), most students regard that it has been most helpful for them to browse other students’ works. 
The mean for this question is as high as 4.53. As high as 93% of the students regard that the quality of their own work can be improved 
by such an emulation through the browsing of others’ works (Item 12). 94% of the users regard that iuhelps to understand the merits 
and shortcomings of other students’ learning by browsing other students’ portfolio (Item 7). 97% of the users regard that it enhances 
their own academic growth and improvement by browsing other students’ portfolios (Item 8). All these survey results have revealed the 
fact that the use of portfolio has really helped the users in learning. 



Table 1: Percentages and means of student agreement in WBLP impacts on learning process 



Impacts on Learning Process 
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Percentages of Agreement 
4 3 2 1 


Means 


1 Set up learning goals may help me with the direction in self-learning 


27 


53 


20 


0 


0 


4.07 


2 The uploading and gathering of course works helps to reveal my genuine learning 
outcomes 


37 




3 


0 


0 


4.33 


3 The writing process for self-reflection and assessment records help me to grasp and 
reflect on genuine learning process 


37 


53 


10 


0 


0 


4.27 


4 Teacher feedback helps me to reflect on my merits and shortcomings in learning 


53 


40 


4 


3 


0 


4.43 


5 The feedback from peers help me to reflect on my merits and shortcomings in 
learning 


47 


43 


7 


3 


0 


4.40 


6 Browsing classmates’ portfolios helps me to reflect on the merits and shortcomings 
in my learning 


47 


43 


7 


3 


0 


4.33 
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7 Browsing classmates’ portfolios helps me to understand the merits and shortcom- 
ings of my classmates in learning 

8 Browsing my classmates’ portfolios helps me to grow and improve in academic 
achievements 

9 Browsing my classmates’ portfolios helps to motivate me for the learning of this 
course. 

10 Browsing my classmates’ portfolios helps me to increase the communication and 
exchanges with my classmates 

1 1 Browsing classmates’ learning goals helps me to understand classmates’ work 
effort, and thus enhances that of my own 

12 Browsing my classmates’ works helps to upgrade the quality of my works 

13 Browsing my classmates’ basic information helps me to understand my classmates 
better 

14 Browsing my classmates’ personal web page helps me to better uiderstand my 
classmates and thus enhances relationships with them 

15 The information in Portfolio Bulletin Board helps me to better understand this 
course 

1 6 The articles of Portfolio Discussion Board help my academic growth 

17 The message feedback \w Portfolio Suggestion Board helps me to solve the prob* 
lems that I have encountered in portfolio creation 

1 8 It helps me to know about the learning outcomes of my classmates by using this 
system 

19 It helps me to know about my teacher feedback and suggestions by using this 
system 

20 It helps me to know about my classmate feedback and suggestions hy using this 
system 

21 It enhances my interactions and exchanges with my classmates by ising this sys^ 
tern 

22 It helps me to better understand my growth and improvements in the course by 
using this system 

23 It helps me to better understand the growth and improvements of other class^ 
mates in this course by using this system 

24 It helps me to learn this course by using this system 

25 I hope to use this system in other courses as well 

Total Means 



47 


47 


3 


38 


59 


0 


43 


50 


7 


37 


43 


17 


57 


33 


7 




30 


4 


50 


33 


14 


40 


40 


13 


57 


33 


7 


17 


SI 


20 


30 


43 


10 




33 


0 




27 


10 




37 


3 


40 


47 


7 


44 


43 


13 


43 


40 


14 




27 


3 




33 


7 


46.96 


41.88 


8.4 



3 0 4.37 

3 0 4.31 

0 0 4.37 

3 0 4.13 

3 0 4.43 

3 0 

3 0 4.30 

7 0 4.13 

3 0 4.43 

6 0 3.83 

17 0 3.87 

0 0 

0 0 E^ 

0 0 ES 

6 0 4.20 

0 0 4.31 

3 0 4.23 

0 0 E^ 

0 0 E^ 

2,76 0 4.33 



The most positive feedback comes out in terms of the overall benefit in using this system (from Item 18 to Item 24), most of 
the students agree that by using this system, it helps them to better understand other students’ learning achievements (mean = 4.67, 
Item 18), allowing them to obtain the feedback and suggestions of other students (mean = 4.57, Item 20), and benefit their learning of 
the course from the portfolio (mean = 4.67, Item 24), 100% of the users regard it useful to utilize the portfolio to get to know the 
learning achievements of other students (Item 18). 97% of the users regard it helpful to obtain the feedback and suggestions of other 
students through this system (Item 20). As high as 97% of the users expressed that it helps them to learn the course by ising this sys- 
tem (Item 24). 

Furthermore, one very interesting phenomenon is when being presented with the question (Item 19 and Item 20) that whether 
using this system helps them to obtain the feedback from teachers (mean = 4.53) or from the other students (mean = 4.57), the results 
show that this system has been more useful for them to obtain the feedback from other students than from their teachers. We deduce 
that one of the reasons may be that students generally hold higher expectation for feedback from teachers. Another reason may be 
that the contents of teachers’ feedback are not as much as that of other students’. Since the teachers have to answer and feedback to 
all students while the students only need to feedback to a few selected peers. The above survey results reflect the fact that the feedback 
from other students has been helpful for them and has thus become the necessary component to help them in learning. 

On the average, as high as 88.84% of the users feel that the system has been helpful in their learning process and outcome. Only 
2.76% a small percentage of the users think otherwise. This shows that the implementation of portfolio truly helps most of the stu- 
dents in academic growth and improvement with great efficiency. Thus it has created positive impact on learning process and outcome. 
Figure 2 shows the distribution of means for student agreement percentages in WBLP impacts on learning process. 
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Figure 2: The distribution of means for student agreement percentages in WBLP impacts on learning process 



To sum up the results of the above user evaluation, the appraisal for impact on learning process is high (mean = 4.33). To learn 
from the lesson above, teachers should give much more encouragement to their students, and pay more attention to the difficulties 
that the students may come across in the implementation process of future portfolio assessment. 



User In-Depth Interviews 

We conducted in-depth interviews with 5 users (who were chosen randomly from the students who had been taking the course) 
after the statistics that have come out based on the suggestion items and questionnaire results to deduce or conclude a few important 
user opinions or suggestions. 

In terms of the impact on learning process, most students regard that the overall design of the WBLP system is good. They are 
able to get to know other students’ learning processes, learning experiences, and thus enhance their learning of the course through their 
browsing. This WBLP system provides the gathering and browsing to access to other students’ works, and thus allows one to get to 
know better about his/her merits and shortcomings. Then one can improve accordingly in learning. Some students regard that to set up 
learning goals may allow themselves to be ready for individual learning direction. However, there seems to be no direct evidence re- 
flected through our assessment standard to support the assumption that such a learning goal set up may help them in terms of assessing 
their final learning results. Hence, its real purpose and function seems to be ambiguous. In addition, introspection and self-assessment 
help to review the process in students’ learning or producing their works. It would allow the students to discover the areas that they 
may improve. As a result, it enhances learning for maintaining such a reviewing process. 



Expert Evaluation 

Upon the completion of this system establishment, we had invited 3 experts (including a portfolio assessment expert, an Inter- 
net technology expert, and the course instructor) to use this system. The researcher was then interviewing them one week later. In this 
way we have obtained the valuable opinions and suggestions of these experts. 

These experts also answer the following questions, such as: “whether this system has been helpful for students’ learning and 
teachers’ teaching?” “Is it possible to provide an effective and appropriate portfolio production and browsing environment in order to 
allow both the teachers and the students to better understand the genuine learning processes and results of students?” Some experts 
regard that this WBLP system provides the opportunity for gathering students’ works for emulation among the peers. Consequently, 
the practice will surely enhance students’ learning result. The introspection and self-assessment records of students would allow the 
teacher to better understand students’ prerequisite ability as well as the practical issues or problems that students have encountered, and 
also students’ achievements in learning. This will help to attain the goal of being more objective in student assessments, and it will also 
help the teacher to adjust his or her teaching method in the future. Some experts point out that the works that students have presented 
via this WBLP system are mainly the final completed works of students, but the processes of such productions in terms of gathering 
and demonstration shall also be incorporated as well. Hence, the production processes or the on-going works shall be demonstrated as 
well. Finally, the system has to improve its management functions in terms of supporting the teachers to perform on-line assessments 
for students’ works, and to track the status of students’ using the services of this system through on-line records and statistics. 



Summaries and Conclusions 

The characteristics of the WBLP is meant to allow the teacher and students’ genuine understanding of the individual learner 
learning process and outcome, and thus to enhance the feedback and interactions among teachers and students. Furthermore, it helps 
students to get to know the merits and shortcoming in his/her own learning, also the difficulties and problems involved, thus to pro- 
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mote the integration of teaching, learning, and assessment. 

As high as 88.84% of the users regard the electronic portfolio system has benefited their learning. As high as 93% of the stu- 
dents regard that the quality of their own work can be improved by the emulation through the browsing of others* works (Item 12). 
94% of the users regard that it helps to understand the merits and shortcomings of other students* learning by browsing other students* 
portfolio. 97% of the users regard that it enhances their own academic growth and improvement by browsing other students* portfo- 
lios. The evaluation results also show that this WBLP system has been more useful for them to obtain the feedback from other stu- 
dents than from their teachers. It reflects the fact that the feedback from other students has been helpful for them and has thus be- 
come the necessary component to help them in learning. 

Whether the use of web*based portfolio system will definitely enhance students* self- regulated learning, self-directed learning, 
enhancing the ability for self-reflection, encouraging learning motivation, etc. are all the interesting issues worthy of our further re- 
search and verification. Therefore, we will conduct the relevant future experiment study in terms of the practical applications in teach- 
ing and learning efficiency for web-based portfolio system. 
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